The task of automated assessment of free-response text passages or essays is distinct from that of scoring short text or multiple choice answers to a series of very specific prompts. Today there are at least 12 programs and associated products, such as the Educational Testing Service's (ETS) e-Rater, PearsonKT's KAT Engine/Intelligent Essay Assessor (IEA), and Vantage Learning's Intellimetric, which are deployed to assess essays as part of self-tutoring systems or as a component of examination marking. Because of the broad potential application of automated assessment to essays, these systems focus as much on assessing the semantic relevance or “topicality” of essays to a given prompt as on assessing the quality of the essay itself.
Many English for Speakers of Other Languages (ESOL) examinations include free text essay-style answer components designed to evaluate candidates' ability to write, with a focus on specific communicative goals. For example, a prompt might specify writing a letter to a friend describing a recent activity or writing an email to a prospective employer justifying a job application. The design, delivery, and marking of such examinations is the focus of considerable research into task validity for the specific skills and levels of attainment expected for a given qualification. The marking schemes for such writing tasks typically emphasize use of varied and effective language appropriate for the genre, exhibiting a range and complexity consonant with the level of attainment desired by the examination. Thus, the marking criteria are not primarily prompt or topic specific, but linguistic. This makes automated assessment for ESOL text (hereafter “AAET”) a distinct subcase of the general problem of marking essays, which in turn suggests a distinct technical approach, if optimal performance and effectiveness are to be achieved.